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Abstract 

A  unifying  framework  for  the  design  and  operation  of  networked  coordinated  systems  has  been  researched 
with  developments  focusing  on  application  in  the  management  of  autonomous  vehicles  which  are  subject 
to  large  external  disturbances,  such  as  wind  gusts.  The  benchmark  problem  is  the  use  of  intervehicle 
communication  and  active  control  to  effect  collision  avoidance  in  multi-vehicle  systems  with  significant 
environmental  disturbances.  The  central  aim  of  the  work  was  to  combine  constrained  optimal  control, 
achieved  via  so-called  Model  Predictive  Control  methods,  with  limited  capacity  communication  link  resource 
assignment  to  achieve  collision  avoidance  with  a  nominal  fleet  formation  and  a  specified  level  of  external 
disturbances.  The  core  results  concern  the  use  of  the  covariance  or  quantified  uncertainty  of  the  estimate 
of  the  other  vehicles’  positions  as  the  link  between  communications  resource  assignment  -  more  bits  of 
communications  means  more  accurate  position  estimates  -  and  collision  avoidance  requirements  -  close 
vehicles  in  the  formation  require  more  accurate  position  information  to  avoid  collision.  A  computational 
tool  is  derived.  The  work  has  been  presented  at  AFRL. 


1  Introduction 


The  work  performed  under  this  grant  has  concentrated  on  the  joint  analysis  of  communication  and  control 
in  achieving  collision  avoidance  of  multiple  autonomous  vehicles  when  afflicted  with  significant  environmental 
disturbances  due  to  wind  gusts  for  example.  A  mathematically  strongly  related  problem  is  that  of  congestion 
control  in  computer  networks,  where  the  vehicles  are  replaced  by  network  nodes,  a  collision  represents  overflowing 
the  buffer  of  a  down-stream  node,  intervehicle  communication  is  interpreted  as  internode  communication  via, 
say,  resource  management  packets,  and  the  disturbances  are  due  to  the  variability  of  traffic  demands  on  the 
network.  The  collision  avoidance  problem  will  proide  the  core  example  of  this  report,  although  the  results  apply 
mutatis  mutandis  to  the  network  congestion  problem,  which  is  an  indication  of  the  fundamental  nature  of  the 
results. 

In  multiple  vehicle  active  collision  avoidance  under  disturbances,  one  uses  the  other  vehicles’  positions  or  pre¬ 
dicted  positions  as  constraints  on  future  controlled  motions.  Thus,  the  underlying  technology  of  the  work  is 
Model  Predictive  Control,  which  is  a  computational  approach  using  constrained  optimization  to  achieve  con¬ 
strained  closed-loop  control  with  strong  connections  between  the  optimization  problem  and  closed-loop  system 
behavior.  Model  Predictive  Control  has  been  an  area  of  considerable  technical  interest  for  over  twenty  years  in 
the  control  community  because  of  its  capacity  to  handle  constraints.  It  is,  however,  formulated  fundamentally 
as  a  full-state  feedback  problem.  That  is,  there  is  no  obvious  provision  for  inaccurate  knowledge  of  the  state  of 
all  the  vehicles.  One  of  the  contributions  of  this  and  preceding  work  of  this  team  has  been  the  incorporation  of 
state  estimates  into  Model  Predictive  Control  [YB05]. 

The  other  core  discipline  underpinning  the  work  is  that  of  state  estimation  or  Kalman  filtering.  Here  the  central 
estimation  problem  is  for  vehicle  i  to  estimate  and  then  predict  ahead  in  time  the  position  of  vehicle  j  from 
limited  data  communicated  from  vehicle  j  over  a  dedicated  communication  channel.  The  central  contribution 
from  this  work  has  been  to  tie  quantitatively  and  computably  the  assignment  of  capacity  to  this  channel  to  the 
eventual  covariance  of  the  subsequent  state  estimate  and  predictions.  This  can  then  be  incorporated  directly 
into  the  feasibility  of  the  constrained  Model  Predictive  Control  problem.  Indeed,  as  shown  in  [KYB08],  this 
feasibility  can  be  included  into  the  single  optimization  problem  of  communications  bandwidth  assignment. 

The  approach  to  collision  avoidance  of  constrained  optimization  provides  further  avenues  for  development  of 
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feasible  solutions  of  rendezvous  problems  and  approach  to  formations  from  large  initial  separations.  The  central 
contribution  of  the  work  undertaken  under  this  grant  has  been  this  adoption  of  constrained  optimization  into 
this  context  and  the  inclusion  of  stochastic  uncertainty. 


2  Model  Predictive  Control  with  State  Estimates 


Model  Predictive  Control  (MPC)  is  an  approach  to  feedback  control  design  which  is  able  to  incorporate  con¬ 
straints.  There  are  two  powerful  ideas  which  lie  at  the  heart  of  the  subject. 


•  At  each  discrete  sample  time  n,  a  constrained  open-loop  (i.e.  no  feedback)  control  problem  is  solved  N 

time  steps  into  the  future,  to  time  n  +  N,  from  the  current  system  state  value  a;„.  This  produces  an 
open-loop  control  sequence  M„+i(a;„), . . .  ,Un+N-i{xn)}  for  N  steps  from  the  current  time.  The 

open-loop  nature  of  the  solution  means  that  all  the  sequence  of  control  values  is  a  function  of  the  current 
state,  Xn,  as  is  made  explicit  in  the  notation.  Because  the  control  solution  is  an  explicit  constrained 
optimization  over  these  Un+j{xn)  values,  it  is  computationally  tractable.  Had  a  closed-loop  solution, 
Un+j{xn+j),  been  sought,  this  would  have  been  in  general  intractable.  However,  of  this  open-loop  solution, 
{ut{xt),ut+i{xt),  ■  ■  ■  ,Un+N-i{xn)},  Only  the  first  control  sample,  is  applied  and  then,  starting 

from  the  resultant  system  state  value,  Xn+i,  one  re-solves  the  same  V-step  constrained  optimization  to 
arrive  at  the  next  open-loop  control  solution  sequence  {un+i{xn+i),Un+2{xn+i),  ■  ■  ■  ,Un+N{xn+i)}  from 
which  only  the  first  control  solution,  t6„+i(a;„+i),  is  applied.  Thus,  a  repeated  solution  of  computationally 
tractable  constrained  open-loop  control  problems  yields  effectively  a  closed-loop  control  sequence  of  first 
steps,  Un+j{Xn+j)- 

•  The  desirable  closed-loop  controlled  system  properties  of  asymptotic  stability  -  a  moot  question  for  a 
finite-time  open-loop  control  problem  -  and  guaranteed  future  feasibility  of  the  sequence  of  open-loop 
solutions  from  the  corresponding  MPC-controlled  initial  states,  Xn+j,  may  be  inferred  from  the  open-loop 
solutions  by  suitable  choice  of  the  terminal  state  constraints  of  the  MPC  constrained  optimization.  In 
particular,  choosing  the  open-loop  terminal  state  constraint  at  time  t  to  be  Xn+N  =  0  implies  that  the 
closed-loop  MPC  solution  will  be  asymptotically  stabilizing.  Thus,  properties  of  the  closed-loop  controlled 
system  may  be  inferred  from  the  statement  of  the  open- loop  problems.  This  property  is  traced  back  to 
Keerthi  and  Gilbert  [KG88]. 


The  archetypal  deterministic  MPC  problem  is  as  follows. 

min  T(  V,  Xyi ,  )  —  Xj^ij^PpjX'n-\-N 

u 

N-l 

T  'y  '  i^n+iQi^'ri+i  T  t 

i=0 

subject  to  :  Xn+i+i  =  f{x„+^,u„+^), 

Xn+i  e  Xi,  {i=l.  ..N), 

Un+i  G  Ui,  (l  =  0,  .  .  .  ,  V  -  1). 

The  minimization  commences  at  time  n  from  initial  state  value  Xn  and  yields  the  V-step  solution  sequence  for 
time  n,  . . . ,  u„+Ar_i}.  The  output  equations  and  constraints  can  be  accommodated  in 

this  formulation,  since  the  output  depends  explicitly  only  on  the  state  and  input.  The  constraints  are,  in  order, 
the  system  dynamics,  the  state  constraints  along  the  V-step  time  horizon,  and  the  control  constraints. 

It  is  clear  from  this  statement  of  MPC  that  these  control  laws  Un+j{xn+j)  are  explicitly  dependent  on  the 
availability  of  the  full  system  state,  Xn+j,  from  measurements.  This  is  often  not  the  case  and  the  full  measure¬ 
ment  needs  to  be  replaced  by  a  state  estimate  produced  by,  say,  a  Kalman  filter  driven  from  available  output 
measurements.  In  the  context  of  coordinated  controlled  systems,  such  as  fleets  of  autonomous  vehicles,  there 
are  several  immediate  observations. 


•  Non-collision  requirements  are  obvious  state  constraints  in  vehicle  control. 

•  The  ‘state’  of  a  multiple  vehicle  fleet  is  the  aggregation  of  all  of  the  individual  states  of  the  vehicles.  If 
environmental  disturbances  are  impinging  on  each  of  the  vehicles,  then  the  predictable  components  of 
these  disturbances  also  form  part  of  each  vehicle’s  extended  state  vector. 

•  In  order  to  form  the  centralized  (effectively  the  all-knowing  air  traffic  controller)  control  solution  for  all 
the  vehicles,  all  of  these  component  substates  need  to  be  known  at  the  centralized  place.  This  requires 
explicit  communication  from  the  sensors  on  each  of  the  vehicles. 

•  For  decentralized,  local  control  solutions,  pairs  of  vehicles  will  need  to  communicate  their  local  state 
information  to  each  other  to  compute  their  collision  avoiding  controls. 

•  The  ability  of  a  vehicle  to  know  its  own  state  is  limited  by  the  accuracy  of  its  sensors  of  position,  ve¬ 
locity,  and  disturbance  knowledge.  The  ability  to  know  another  vehicle’s  state  is  further  limited  by  the 
communication  resource  assigned  to  that  communication,  nominally  the  bit  rate. 

•  Accordingly,  self-state  values  and  other-state  values  are  necessarily  inexact  and  need  to  be  replaced  by 
state  estimates.  The  MFC  control  law  needs  to  be  adjusted  to  accommodate  these  inaccuracies.  We 
distinguish  two  separate  kinds  of  state  estimators;  self-state  estimators  of  the  vehicle’s  own  state,  and 
cross- estimators  of  other  vehicles’  states. 

•  The  quantification  of  uncertainty  or  inaccuracy  in  state  estimates  is  normally  probabilistic  in  nature  and 
is  captured  by  second  order  statistics  via  the  covariance  function. 


In  probabilistic  terms,  the  original  MFC  problem  to  be  solved  at  vehicle  j  may  be  replaced  by 

mm 

N-1 

+  '  {Xn+jQjXn-i-i  + 
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subject  to  : 

(*=l...iV), 

Un+i  G  Ui,  {i  =  0,. . .  ,N  -  1). 
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Here:  the  actual  vehicle-disturbance  state  and  position  is  replaced  by  their  self-estimates  and  yf^^i,  the 
other  vehicle-disturbance  state  and  position  is  replaced  by  their  local  cross-estimates  x^j^n+i  and  the 

non-collision  constraint  that  the  positions  be  separated  by  a  fixed  amount  a  is  replaced  with  a  probabilistic 
statement  of  separation  with  high  probability  1  —  e.  Since  the  estimates  xf^+i  n+i  i®  available 

at  vehicle  j,  this  has  translated  the  original  MFC  problem  to  a  feasible  form. 

In  the  case  that  the  dynamics  are  linear  and  the  probability  distributions  gaussian,  the  probabilistic  non-collision 
constraint,  F  (^\y^„_^.^  —  jjjn+il  <  <  £>  may  be  transformed  into  a  deterministic  form. 

Lemma  1  ([KYB08]).  Assume  that  all  estimation  errors  are  gaussian  with  self-  and  cross-  covariances  given  by 
'^h+j  0,'nd  respectively.  Denote  the  dimension  of  the  position  vector,  j/„+i  =  Cxn+i,  by  d  and  denote  the 

cumulative  distribution  function  of  the  density  with  d  degrees  of  freedom  by  'l'd(-)-  Consider  the  probabilistic 
no-collision  constraint  (1)  with  weighting  matrix  M  >  0, 

F  {\yn+j  -  yi+j\M  <  a)  <  e.  (2) 

and  define  the  value  (3  to  be  any  value  satisfying 

«'d(/3")  >  1  -  e. 
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Then,  with 
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satisfaction  of  the  probabilistic  no-collision  constraint  is  implied  by 


Vn+jl  n  ^i,n+j|n 
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(4) 


This  is  one  of  the  key  technical  observations  of  the  research.  This  is  Lemma  1  from  [KYB08].  The  central 
achievement  is  the  translation  of  the  probabilistic  statement  concerning  the  unknown  states  into  a  deterministic 
statement  about  the  known  state  estimates.  The  covariances  of  the  estimates,  S,  enter  as  the  pivot  of  this 
transformation.  Accordingly,  the  management  of  communication  resources  is  formulated  as  in  terms  of  the 
specification  of  covariances  of  cross-estimates,  since  the  greater  the  communication  resource  the  smaller  the 
estimate  covariance. 

For  a  specific  vehicle  target  fleet  formation,  where  vehicles  i  and  £  are  given  prescribed  nominal  (undisturbed) 
positions  y\  and  yf,  then  the  non-collision  constraint  becomes  a  constraint  on  the  estimate  covariances.  This 
constraint  is  based  on  the  operational  requirement  that  under  the  action  of  disturbances  the  constraints  are 
usually  inactive.  This  restriction  on  the  estimate  covariance  is  a  function  of  the  formation  geometry  positions 
and  the  underlying  disturbance  field. 

Lemma  2  ([KYB08]).  The  {i,£)  no-collision  constraint  is  usually  inactive  provided  the  cross-estimator  covari¬ 
ance  satisfies 
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3  Communications  modeling  and  assignment 


With  this  statement  of  the  design  requirement  for  the  cross-estimator  (5),  which  we  state  succinctly  as  <  W, 
the  next  stage  is  to  interpret  this  in  terms  of  the  implied  specifications  on  communications  between  vehicles. 
The  details  of  this  process  are  the  province  of  [KYB08],  but  which  we  rapidly  summarize  here. 

The  core  idea  is  that  the  communications  bit-rate  assignment  causes  quantization  errors  on  the  transmitted  self¬ 
state  estimate  and  future  control  values  sent  from  vehicle  £  to  vehicle  i  and  used  as  the  basis  of  a  future  prediction 
of  position  as  part  of  the  non-collision  constraint.  If  a  value  is  truncated  to  P  bits,  then  the  quantization  error 
is  white  and  uniformly  distributed  between  (—2“^,  2“^)  times  the  dimension  of  the  underlying  area  of  interest. 
The  associated  covariance  is  simply  computed.  This  ties  covariance  to  communications  bit  rate. 

To  add  some  further  unexplained  detail  from  [KYB08],  the  following  equations  describe  the  evolution  of  the 
cross-estimator  of  the  vehicle-disturbance  state  of  vehicle  £  as  computed  at  vehicle  i  in  the  case  of  linear  vehicle 
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The  stationary  covariance  calculation  of  this  augmented  system  is 

V  =  AVA^  +  GiQGj  +  G2 
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where  Q  =  cov( 


^  .  Here  is  the  quantization  noise  of  the  transmission  of  the  current  self-state  estimate 

Vn—1 

from  £  to  i  and  ?72^  to  ^  are  the  quantization  errors  of  the  transmission  of  the  control  values.  From  (8),  it 
is  clear  that  the  stationary  covariance  of  the  state  is  a  simple  function  of  these  quantization  error  covariances. 
The  value  K  is  the  Kalman  filter  gain  of  the  cross-estimator. 

If  we  denote  the  bit-rate  assigned  to  the  state  value  as  niA  and  that  assigned  to  the  control  as  mA  then  we 
have  the  following  theorem  from  [KYB08]. 

Theorem  1.  Any  solution  {P  =  Y~',K  =  PY K,ml.,mA}  to  the  following  set  of  Linear  Matrix  Inequalities, 
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where  L  is  the  total  bit-rate  resource  and  W  =  — — A - M~'  —  CY\ 


-p - M~'  —  ,  yields  a  simultaneous  solution  for 

steady-state  cross- estimator  covariance  P,  cross- estimator  filter  gain  matrix  K,  and  communication  hit  rates 
mi  and  mi,. 


The  thrust  of  this  section  is  to  point  out  that  the  determination  of  communication  bit  rate  assignments  is  incor¬ 
porated  into  the  analysis  using  simple  tools  from  Kalman  filtering  and  computational  techniques  which  are  part 
of  standard  packages,  such  as  matlab.  The  connection  to  non-collision  constraints  and  Model  Predictive  Control 
is  via  the  covariance  limit  (11).  This  linkage  between  control  constraints  and  communications  requirements  with 
computational  tools  is  the  primary  contribution  of  this  research  project. 


X  position(m) 


Figure  1:  [KZB08]  Two  vehicle  trajectories  (Vehicle  1  solid  line,  Vehicle  2  dotted  line)  under  wind  gusts  with 
collision  avoidance  and  control  constraints.  Axes  indicate  positions  and  curve  are  trajectories  parametrized  by 
time  with  times  indicated.  The  upper  curve  shows  the  commencement  of  the  gust  with  the  vehicles  at  their 
target  positions  through  to  the  gust’s  acme  and  the  lower  plot  the  denouement  and  controlled  return  to  position. 

4  Further  developments 


The  development  of  these  theoretical  and  computational  connections  between  control,  constraints,  disturbances, 
and  communications  resource  assignment  are  most  easily  developed  and  understood  in  the  linear  gaussian  case, 
where  the  methods  are  explicit  and  non-conservative.  In  more  realistic  scenarios  of  nonlinear  systems  (which  is 
implicit  in  constrained  control),  the  approach  still  provides  guidance  and  computational  tools.  Indeed,  compu¬ 
tations  contained  in  the  paper  [KZB08]  illustrate  the  closeness  of  the  approximation  in  a  nonlinear  environment 
with  realistic  wind-gust  disturbances.  The  two-dimensional  collision-avoiding  constrained  trajectories  for  two 
vehicles  under  significant  wind- gusts  in  shown  in  Figure  1. 

The  analytical  approach  has  been  centered  on  the  stationary  case,  where  the  vehicles  are  moving  about  fixed 
target  locations  in  a  formation  and  being  disturbed  by  statistically  stationary  external  forces.  For  the  problem 
of  rendezvous,  where  a  fleet  of  vehicles  is  required  to  enter  a  formation  from  distant  locations  without  collision, 
the  Model  Predictive  Framework  still  provides  access  to  tools  but  the  feasibility  of  the  methods  requires  more 


technical  machinery.  This  is  presented  in  [KB07,  KB08,  KB09]  with  the  latter  two  references  describing  the  use 
of  Control  Lyapunov  Functions  to  permit  MFC  to  improve  control  performance  over  finite  horizons. 

The  reliance  of  these  approaches  on  communications  as  the  sole  means  of  estimating  other  vehicles’  positions 
fails  to  incorporate  the  use  of  active  sensors,  such  as  radar.  However,  the  framework  of  estimate  covariance 
management  admits  a  simple  inclusion  of  sensor  information.  Indeed,  much  of  the  study  of  radar  systems 
is  itself  based  on  management  of  position  and  velocity  estimate  covariances.  The  fusion  of  multiple  sources 
of  information  via  covariance  computation  is  well  studied  field.  Very  recent  work,  not  part  of  this  grant  but 
following  from  it,  has  focused  on  making  these  statements  precise.  In  the  situation  of  dynamically  varying 
vehicle  formation  geometries,  such  as  detection  and  then  inclusion  of  a  new  vehicle  into  a  formation,  these 
sensor-and-communication  based  methods  are  shown  to  work  simply.  Additionally,  ideas  from  cellular  phone 
systems  concerning  control  and  traffic  channels,  call  admission,  etc,  have  provided  elegant  approaches  to  vehicle 
inclusion  and  departure  as  well  as  to  obstacle  avoidance  and  evasion.  This  work  is  ongoing. 


5  AFRL  visit 


Efforts  to  promulgate  the  research  supported  by  this  grant  have  been  publication  in  a  number  of  journal  and 
conference  papers  and  a  book  chapter;  see  Appendix.  The  work  was  presented  and  discussed  at  some  length 
during  a  visit  of  the  Principal  Investigator  to  the  Air  Force  Research  Laboratories  in  Dayton  OH  in  June,  2009. 
The  host  was  Dr  Derek  Kingston. 
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The  communications  resource  assignment  is  formulated  as  a  routine  constrained  optimization  calculation  using 
Linear  Matrix  Inequalities.  If  a  feasible  solution  exists,  it  yields  the  communications  bit-rate  allocation  between 
vehicles,  Kalman  filtering  estimator  gains,  and  limiting  covariance  values. 


7  Theses  arising  from  this  work 


Two  UGSD  theses  contain  the  bulk  of  the  work. 


Jun  Yan,  Constrained  Model  Predictive  Control,  State  Estimation  and  Coordination,  University  of 
Galifornia,  San  Diego,  April  2006. 


This  thesis,  supported  by  an  NSF  grant,  includes  the  early  developments  of  including  state  estimates,  with  their 
attendant  uncertainty  quantified  by  the  covariance,  into  Model  Predictive  Gontrol.  The  predominant  target 
problem  is  that  of  network  congestion  control. 


Keunmo  Kang,  Information  in  Coordinated  System  Control,  University  of  California,  San  Diego, 
September  2008. 


This  thesis,  supported  in  its  entirety  by  this  AFOSR  grant,  develops  a  number  of  themes  from  the  covariance- 
based  adjustment  to  constraints  presented  by  Yan.  These  include;  the  formulation  of  the  communications 
resource  assignment  for  static  vehicle  formations,  approaches  for  the  guaranteed  feasibility  of  initial  rendezvous 
to  the  target  formation  based  on  polytopic  uncertainty  sets,  and  stable  transition  to  formation  from  large 
separation  based  on  Control  Lyapunov  Functions. 

Additionally,  Masters  student  David  Zhang  was  involved  in  the  development  of  a  comprehensive  simulation  of 
the  methods  with  realistic  wind  gusts  and  coordination  between  two  vehicles  with  the  minimal  communication 
rates.  This  work  was  presented  in  [KZB08]. 
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